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1.  Introduction 


1.2  Background 

For  the  past  5  years  (2013-2017),  a  team  led  by  personnel  from  the  US  Army 
Research  Laboratory’s  (ARL’s)  Human  Research  and  Engineering  Directorate 
(HRED)  Field  Element  at  Fort  Bliss,  Texas,  has  provided  Human-Systems 
Integration  (HSI)  support  to  the  Army’s  Brigade  Modernization  Command  (BMC) 
for  the  Network  Integration  Evaluations  (NIEs)*.  The  NIEs  are  a  series  of 
semiannual,  brigade-level  exercises  intended  to  integrate  and  mature  the  Army’s 
tactical  networks  in  an  operational  context.  The  broad  scope  of  the  NIEs  pennits 
human  performance  effects  and  HSI  issues  to  be  assessed  at  the  individual  Soldier- 
system  level,  as  well  as  at  the  system-of-systems  and  unit  levels.  In  this  respect, 
network-enabled  command  posts  (CPs)  are  treated  as  complex  sociotechnical 
systems  of  systems  embedded  within  a  multi-echelon  unit  context.  A  sociotechnical 
system  is  a  work  system  consisting  of  people  interacting  with  a  technology  suite 
intended  to  accomplish  a  specific  organizational  function.  In  the  case  of  CPs,  that 
organizational  function  is  mission  command. 

The  initial  focus  of  ARL’s  HSI  support  was  the  cognitive  load  associated  with 
network-enabled  mission  command.  Simply  stated,  cognitive  load  is  defined  as  the 
aggregate  mental  load  placed  on  multi-echelon  commanders  and  key  battle  staff 
personnel  by  an  increasingly  complex  mission  command  work  setting.  As  a 
construct  impacting  mission  command,  cognitive  load  is  developed  in  additional 
detail  in  Hawley  (2014).  An  additional  opportunity  provided  by  the  NIEs  was  the 
ability  to  address  the  nonmateriel  modifications  (e.g.,  Doctrine,  Organization, 
Training,  Leadership  and  Education,  and  Personnel  [DOTLP])  necessary  to  make 
effective  use  of  the  systems  underpinning  network-enabled  mission  command. 
Previous  force-modernization  research,  along  with  results  from  the  ARL  team’s 
NIE  support  work,  suggests  that  robust  DOTLP  adaptations  are  on  a  par  with 
materiel  as  contributors  to  enhanced  mission  command  performance  and  unit 
effectiveness  (Gonzales  et  al.  2005).  Unfortunately,  DOTLP  developments  and 
modifications  often  take  a  back  seat  to  equipment-related  concerns  during  the 
development,  testing,  and  fielding  of  new  materiel.  Units  receiving  new  materiel 
are  often  left  “on  their  own”  to  determine  how  to  best  use  that  equipment  to  meet 
mission  objectives.  The  result  of  this  lack  of  emphasis  on  DOTLP  issues  early-on 


*  Following  NIE  16.2,  the  first  exercise  in  each  NIE  sequence  was  redesignated  as  an  Army 
Warfighting  Assessment  (AW A).  Following  AWA  17.1,  the  BMC  was  redesignated  as  the  Joint 
Modernization  Command.  AWAs  were  redesignated  as  Joint  Warfighting  Assessments,  and  NIEs 
were  redesignated  as  Capability  Integration  Evaluations. 
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can  be  misjudgments  concerning  the  potential  military  utility  of  new  systems  and 
technology  as  well  as  the  nature  of  the  DOTLP  package  required  to  adequately 
support  those  systems. 

Based  on  results  across  NIEs,  the  ARL/HRED  team  provided  specific  mitigation 
recommendations  in  3  broad  areas  affecting  CP  and  mission  command  complexity 
as  well  as  associated  cognitive  load:  1)  mission  command  system  design  (e.g., 
ergonomics)  and  component  integration,  2)  knowledge  management  (KM),  and  3) 
individual  and  team-oriented  battle  staff  training  (Hawley  2015;  Hawley  and 
Swehla  2016).  Mission  command  component  integration  was  further  divided  into 
interoperability  effects  and  operational  integration  challenges.  Operational 
integration  refers  to  incorporating  new  materiel  solutions  into  CP  and  mission 
command  processes  and  procedures  as  well  as  adjusting  those  processes  and 
procedures  to  reflect  the  capabilities  of  new  technologies.  New  technologies  often 
change  the  nature  of  the  work  processes  they  are  intended  to  support  (see,  for 
example,  Wickens  et  al.  2013).  Individual  Soldiers,  functional  teams,  and  units 
must  adapt  to  the  new  capabilities  they  now  possess. 

Results  across  NIEs  indicated  that  the  factors  previously  cited  combine  and  act  to 
increase  the  aggregate  level  of  perceived  complexity  and  cognitive  load  on  CP 
personnel.  The  mission  command  role  itself  is  intrinsically  complex  and 
demanding.  However,  a  work  setting  with  a  large  number  of  design-related  “rough 
edges”  will  give  the  impression  of  being  more  complex  and  intimidating  than  one 
that  has  been  better  designed  and  integrated.  While  some  of  the  cognitive  load 
associated  with  mission  command  in  NIE  CPs  is  intrinsic  to  battle  staff  roles,  high 
levels  of  extraneous  cognitive  load  are  needless  consequences  of  insufficient 
attention  to  HSI  in  component  design  and  integration  coupled  with  inadequate 
training  for  both  individual  system  users  and  for  battle  staffs  operating  as  a  team. 
As  used  here,  the  tenn  “inadequate  training”  refers  to  training  that  is  1)  too  short  to 
produce  necessary  levels  of  Soldier  competence,  2)  ill-focused  in  the  sense  that 
training  content  does  not  address  critical  individual  or  team  skills,  or  3) 
inappropriate  in  that  the  instructional  methods  used  are  not  suitable  for  the  job’s 
skill  content  or  required  level  of  proficiency. 

“Battle  staff  integration”  is  a  tenn  used  to  characterize  the  process  by  which  the 
individuals  comprising  a  unit’s  battle  staff  leam  to  work  together  as  an  effective 
mission  command  team  (Olmstead  1992).  Inadequate  individual  and  team-oriented 
training,  staff  member  inexperience,  and  rapid  personnel  turnover  within  battle 
staffs  can  lead  to  a  situation  in  which  effective  battle  staff  integration  is  not 
achieved,  and  this  failure  impacts  mission  command  performance  and  unit 
effectiveness  (Thompson  et  al.  1991;  Sauer  1996). 
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Failure  to  address  network-enabled  CPs  as  systems  of  systems  that  support  mission 
command  as  an  integrated  warfighting  function  was  another  contributing  factor  in 
perceived  complexity  and  extraneous  cognitive  load.  A  CP  is  a  system  in  and  of 
itself  (i.e.,  a  system  of  systems).  However,  the  components  comprising 
modernizing  CPs  often  were  developed  and  evaluated  mostly  in  isolation  and  by 
different  proponents  and  vendors.  Their  relationship  with  other  CP  components  was 
not  always  considered,  and  their  design  was  not  based  on  an  understanding  of 
complex  cognitive  work  in  context.  Consequently,  the  pieces  of  the  CP  “puzzle” 
did  not  always  fit  together  smoothly  or  comprehensively  to  support  mission 
command  as  an  integrated  warfighting  function.  Battle  staff  members  were  often 
required  to  compensate  for  both  component  design  inadequacies  vis-a-vis  role 
requirements  and  integration  shortfalls.  These  deficiencies  contributed  to  perceived 
system  complexity  and  drove  extraneous  cognitive  load.  This  “stove-piping” 
problem  is  not  unique  to  mission  command  systems  or  to  the  Anny.  Many  of  these 
same  deficiencies  have  also  been  noted  in  reports  from  independent  sources  such 
as  the  Mission  Command  Battle  Laboratory  (see,  for  example,  MCCTS  2016). 

ARL/HRED’s  HSI  support  team  first  identified  KM  deficiencies  affecting  mission 
command  performance  and  contributing  to  cognitive  load  during  NIE  14.2.  The 
support  team  followed  up  and  confirmed  those  initial  observations  during  NIE  15.1. 
During  NIE  15.2,  the  team  was  joined  by  a  subject  matter  expert  (SME)  from  the 
Anny  KM  Proponent’s  office  at  the  Army  Mission  Command  Center  of  Excellence 
(MCCoE).  The  team  once  again  confinned  observations  regarding  KM  deficiencies 
and  their  effect  on  mission  command.  Potential  paths  forward  for  improving  KM 
perfonnance  in  CPs  also  were  identified  (Hawley  and  Swehla  2016).  The  test  unit 
during  all  of  these  NIE  events  was  the  2nd  Brigade  Combat  Team  (BCT)  of  the  1st 
Annored  Division  (2/1  AD). 

1.2  Knowledge  Management 

The  Army  defines  KM  as  the  process  of  enabling  knowledge  flow  to  enhance 
shared  understanding,  learning,  and  decision  making  (HDA  2012b;  HDA  2014). 
Applying  effective  KM  practices  ensures  that  knowledge  products  and  services  are 
relevant,  accurate,  timely,  and  usable  for  commanders  and  decision  makers 
(HDA  2012a).  Effective  KM  enables  commanders  to  make  infonned,  timely 
decisions  despite  the  uncertainty  of  operations  (ADRP  3.0  2012).  An  information 
paper  on  training  for  mission  command  produced  by  the  MCCoE  characterizes  KM 
as  the  “Binding  Idea”  underlying  effective  CP  operations  (TMCWF  2013). 

The  cognitive  demands  associated  with  KM  are  not  trivial,  particularly  in  a 
data-intensive  CP  setting.  Advancements  in  network  technology  have  resulted  in  a 
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shift  from  not  having  all  the  infonnation  necessary  to  support  operational  decisions 
to  having  an  overwhelming  amount,  where  crucial  information  can  be  lost  within 
the  immense  amount  of  data  available.  Results  across  NIEs  strongly  suggest  this 
situation  has  led  to  an  increased  level  of  perceived  mission  command  complexity 
and  cognitive  load  for  commanders  and  their  battle  staffs.  For  example,  the  tenn 
most  often  used  by  NIE  participants  to  characterize  cognitive  load  was  “information 
overload.”  As  one  company  commander  put  it,  “We  have  too  much  information  to 
be  processed  in  the  time  allowed.  It’s  too  much  for  one  person  to  handle. 
Sometimes,  I  don’t  know  what  I’m  missing.”  High  levels  of  cognitive  load  can  lead 
to  increased  performance  times  and  error  rates,  and  a  decreased  likelihood  of 
successfully  accomplishing  a  task.  Effective  KM  practices  are  thus  an  important 
contributor  to  moderating  extraneous  cognitive  load  and  enhancing  performance  in 
tactical  CPs. 

Observations  from  NIEs  indicated  that  inadequate  KM  practices  in  a  tactical  unit 
might  not  show  up  during  routine  unit  operations.  On  the  surface,  a  unit  might 
appear  to  manage  information  adequately,  but  KM  processes  are  neither  explicitly 
defined,  standardized,  and  integrated  into  new  mission  command  systems,  nor  are 
they  rigorously  applied  across  unit  echelons.  Incidents  attributable  to  KM 
deficiencies  typically  emerge  when  1)  an  unexpected  event  forces  the  unit  out  of 
routine  practices,  2)  information-handling  deficiencies  impede  operations,  and  3) 
unit  performance  is  negatively  impacted.  Battle  staff  personnel  end  up  “chasing 
relevant  data”  to  the  detriment  of  effective  mission  command  perfonnance.  Several 
such  incidents  attributable  to  inadequate  KM  practices  were  observed  and 
documented  during  the  NIEs.  One  of  the  best  documented  examples  of  KM 
deficiencies  and  their  impact  on  mission  command  operations  occurred  during  NIE 
14.2.  This  example  was  initially  cited  in  Hawley  (2015)  and  is  repeated  here  to 
illustrate  the  importance  of  KM  to  effective  mission  command. 

The  following  remarks  were  extracted  from  an  Observer/Controller  (O/C)  Drop 
Card  from  the  brigade  CP  dated  14  May  2014,  along  with  a  supporting  comment 
from  a  post-NIE  command-level  focus  group  session: 

The  problem  is,  how  does  the  TOC  [Tactical  Operations  Center]  manage 
infonnation  and  create  understanding?  . . .  The  BDE  [brigade]  TOC  continues  to 
struggle  analyzing  and  distributing  information  that  flows  into  the  TOC.  . . .  This 
is  a  systems  management  issue.  There  is  not  any  cross  communication  of  intel 
across  the  TOC  floor.  The  BDE  S-2  may  collect  it  [intel  data],  but  the  BC  [battle 
captain]  is  not  populating  anything  on  the  COP  [Common  Operating  Picture]  to 
establish  SA  [Situation  Awareness].  ...  I  cannot  help  but  think  that  the  BDE  CDR 
[commander]  is  frustrated  with  his  intel  and  assisted  understanding  of  what  is 
occurring  in  the  battle  space.  . . .  The  BDE  is  piecemealing  the  fight  [each  mission 
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command  component  within  the  CP  is  operating  semi  autonomously  with  little 
overall  coordination  by  the  battle  staff].  The  network  is  pushing  information,  but 
the  TOC  is  being  overwhelmed  with  information.  A  lack  of  information 
management  from  the  network  is  causing  frustration.  . . .  Information  must  be  better 
leveraged  to  make  decisions  more  rapidly. 

The  following  is  an  independent  supporting  comment  from  a  post-NIE  focus  group 
session: 

We  have  the  technology/capability  but  can’t  seem  to  figure  out  how  to  do  it;  we 
struggle  to  get  a  picture  from  platform  to  platform.  How  do  we  solve  this?  The 
network  has  tremendous  capability  but  is  incapable  of  being  leveraged  by  BDE 
and  below  to  its  fullest  capacity;  for  4  NIEs,  we  have  struggled  to  get  relevant 
intelligence  to  the  CO  [commanding  officer]  and  below.  ...  We  [require] 
components  that  enable  staff  to  support  the  CDR  intellectually. 

This  Drop  Card  is  an  insightful  comment  from  an  experienced  O/C  characterizing 
the  roots  of  mission  command  dysfunction  in  the  brigade  CP  from  the  perspective 
of  complexity  and  cognitive  load.  Many  issues  are  implied  in  these  brief  comments, 
but  one  stands  out  in  particular:  inadequate  KM  skills  on  the  part  of  the  battle  staff, 
particularly  the  battle  captain.  Simply  stated,  KM  is  characterized  as  the  process  by 
which  data  are  transformed  into  information  (data  in  context),  and  information  is 
then  transformed  into  knowledge  that  can  be  used  to  support  command  decision 
making  (i.e.,  to  “support  the  CDR  intellectually”).  The  cognitive  processes  that 
underlie  KM  include  collecting,  organizing,  and  summarizing  incoming  data  to 
form  information  (data  in  context).  Information  is  analyzed  and  synthesized  to 
support  knowledge  “creation”.  The  final  step  in  the  KM  process  is  command 
decision  making  (Leistner  2010). 

The  implications  of  the  previous  comments  are  clear.  KM  was  not  being  performed 
effectively  in  the  brigade  CP.  Consider  critical  supporting  remarks  in  turn:  The 
network  is  pushing  information  (data);  the  TOC  is  overwhelmed  with  information 
(raw  data);  the  brigade  battle  captain  is  not  populating  anything  on  the  COP  to 
establish  SA  (data  are  not  being  organized  and  summarized  into  usable  information; 
information  analysis  and  synthesis  are  not  being  performed);  a  lack  of  information 
management  from  the  network  is  causing  frustration.  And  finally  a  capstone  remark 
in  the  Drop  Card  from  the  brigade  commander:  “For  4  NIEs,  we  have  struggled  to 
get  relevant  intelligence  to  the  CO.  ...  We  [require]  components  that  enable  staff 
to  support  the  CDR  intellectually”  (Hawley  2015). 

Follow-up  investigations  during  subsequent  NIEs  indicated  that  2/1  AD  did  not 
have  a  comprehensive  KM  program,  and  key  battle  staff  personnel  lacked  formal 
KM  training  (Hawley  and  Swehla  2016).  Moreover,  KM-trained  personnel 

Approved  for  public  release;  distribution  is  unlimited. 

5 


assigned  to  the  unit  frequently  were  not  used  in  their  doctrinally  defined  roles. 
Similar  observations  regarding  KM  implementation  in  tactical  units  are  not  unique 
to  2/1  AD,  but  have  shown  up  in  other  units  during  Combat  Training  Center  (CTC) 
rotations  and  other  mission  command-related  events  (MCTPTDAWE  2014; 
MCCTS  2016).  These  results  from  NIEs  and  from  other  relevant  exercises  led 
directly  to  an  applied  KM  demonstration  project  involving  2/1  AD  as  part  of  Anny 
Warfighting  Assessment  (A WA)  17.1. 

1.3  Purpose 

This  report  presents  results  from  the  applied  KM  demonstration  project  conducted 
in  2/1  AD  immediately  prior  to  and  during  AWA  17.1.  The  focus  was  two-fold. 
The  first  objective  was  to  assess  the  effectiveness  of  the  Anny  KM  Proponent’s  3 
doctrine-based  KM  courses:  Knowledge  Management  Qualification  Course 
(KMQC),  Knowledge  Management  Representative  Course  (KMRC),  and  Senior 
Leader  Executive  Overview  (SLEO).  The  second  objective  was  to  assess  the  utility 
of  the  KM  Proponent’s  KM  Maturity  Model  as  a  tool  for  commanders  and  their 
senior  staffs  to  assess  KM  maturity  within  their  unit  and  detennine  concrete  paths 
forward  for  KM  improvements.  Specific  assessment  questions  nested  under  these 
overall  objectives  included  the  following: 

•  How  effective  is  the  KMQC  in  preparing  KM  Officers  (KMOs)  to  execute 
the  duties  and  responsibilities  of  a  KM  professional? 

•  How  effective  is  the  KMRC  in  preparing  staff  KM  Representatives  (KMRs) 
to  support  the  KMO  in  conducting  unit  KM  operations? 

•  How  effective  is  the  SLEO  in  ensuring  that  senior  leaders  understand  KM 
fundamentals  and  how  it  enables  mission  command? 

•  Does  the  KM  Maturity  Model  enable  the  unit  to  fully  assess  unit  KM 
posture? 

•  How  does  formalized  training  (KMQC,  KMRC,  and  SLEO)  contribute  to 
shared  understanding  throughout  the  operations  process? 

Each  of  these  KM  courses  and  the  KM  Maturity  Model  are  briefly  described  in  the 
following. 

Knowledge  Management  Qualification  Course.  The  KMQC  is  a  3-week,  120-h 
course  intended  to  provide  officers,  noncommissioned  officers  (NCOs),  and 
Department  of  the  Anny  (DA)  civilians  assigned  to  KM  positions  the  education 
and  training  in  the  core  skills  necessary  to  successfully  perform  KM  duties.  The 
course  is  intended  to  prepare  Soldiers  to  perfonn  KM  Section  Officer  and  Soldier 
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responsibilities  in  support  of  mission  command  execution.  The  course  emphasizes 
Anny  KM  doctrine  and  enabling  tactics,  techniques,  and  procedures  (TTPs).  It  is 
intended  to  prepare  graduates  to  be  effective  KM  leaders  in  a  variety  of 
organizations.  Graduates  should  be  able  to  lead  KM  coordination,  planning,  and 
execution  in  a  range  of  positions. 

Knowledge  Management  Representative  Course.  The  KMRC  is  a  20-h  course 
provided  by  an  instructor  team  from  the  Army  KM  Proponent’s  Office.  The  team 
presents  the  course  at  a  location  of  the  target  organization’s  choosing.  The  course 
is  intended  to  provide  designated  unit  personnel  with  the  knowledge  and  skills 
necessary  to  serve  as  KMRs  within  their  organization.  The  KMRC  focuses  on  the 
components  of  KM — people,  organization,  processes,  and  tools — and  their 
interaction  to  support  mission  command  within  an  organization.  The  course  focuses 
on  pairing  the  subject  matter  expertise  of  Soldiers  with  KM  best  practices  to 
enhance  mission  command  and  organizational  effectiveness. 

Senior  Leader  Executive  Overview.  The  SLEO  is  a  1-  to  3-h  KM  overview  for 
the  senior  leaders  of  brigade  and  higher  echelon  organizations.  The  course  focuses 
on  the  importance  of  leveraging  KM  assets  within  their  organizations  and  the 
application  of  Army  KM  best  practices  to  improve  mission  command  and 
organizational  perfonnance.  The  SLEO  is  provided  in  conjunction  with  the  KMRC. 

KM  Maturity  Model.  The  KM  Maturity  Model  is  a  computer-based  tool  (i.e.,  a 
spreadsheet)  intended  to  assist  organizations  in  assessing  the  maturity  of  their  KM 
program.  The  tool  is  intended  to  provide  a  way  to  evaluate  organizational  “efforts” 
or  KM  activities  to  include  people,  processes,  and  tools  using  a  simple  1-5  Likert 
rating  scale.  The  efforts  listed  in  the  tool  are  based  on  doctrinal  requirements  or 
emerging  KM  best  practices  and  are  linked  to  mission  command  effectiveness.  The 
KM  Maturity  Model  is  intended  to  guide  the  KMO  and  KM  Working  Group 
(KMWG)  in  their  initial  development  of  an  applied  KM  program  in  a  tactical  unit. 
The  KM  maturity  Model  also  is  potentially  useful  in  tracking  a  program’s  progress 
over  time.  Appendix  A  provides  an  overview  of  the  KM  Maturity  Model.  The 
material  in  Appendix  A  was  provided  by  the  Army  KM  Proponent. 

2.  The  Applied  KM  Demonstration  Project  in  2/1  AD 


2.1  Program  Initiation 

All  3  KM-related  courses  were  conducted  with  2/1  AD  Soldiers  during  the  summer 
of  2016,  immediately  prior  to  the  start  of  AWA  17.1.  The  unit  appointed  a  KMO 
(2/1  AD’s  Network  Services  Technician,  a  Chief  Warrant  Officer  2  [CW2,  holding 
Military  Occupational  Specialty  [MOS]  255A),  who  attended  the  3-week  KMQC 
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presented  by  the  KM  Proponent  at  Fort  Leavenworth,  Kansas.  The  brigade  also 
identified  and  appointed  KMRs  for  each  of  2/1  AD’s  subordinate  tactical  units.  The 
Anny  KM  proponent  provided  the  KMRC  and  SLEO  to  unit  staff  and  leaders 
(battle  captains,  operations  NCOs,  battle  NCOs,  etc.)  from  across  2/1  AD  on-site  at 
Fort  Bliss,  Texas,  during  the  period  23-25  August  2016.  Unit  personnel  attending 
the  KMRC  also  conducted  a  baseline  application  of  the  KM  Maturity  Model. 

Following  the  KMRC,  the  unit’s  operational  tempo  and  other  activities  associated 
with  the  run-up  to  AWA  17.1  limited  any  KM -related  start-up  work  the  KMO  had 
planned.  For  example,  the  KMO  was  busy  full-time  in  the  AWA  17.1  Integration 
Motor  Pool  until  very  near  the  start  of  the  exercise.  The  KMO  later  remarked  that 
he  did  not  think  he  was  the  right  person  to  fill  that  role,  noting  that  he  was  not 
familiar  enough  with  mission  command  staff  operations  to  function  effectively  as 
the  unit’s  KMO.  Rather,  he  stated  that  the  KMO  should  be  a  person  from  the 
operations  and  training  (S-3)  section  of  the  brigade’s  staff.  The  KMO  also 
commented  that  serving  in  that  role  is  a  full-time  job  and  should  not  be  viewed  as 
a  role  that  can  be  performed  in  combination  with  another  full-time  job.  This  is 
particularly  relevant  during  the  applied  KM  program’s  start-up  phase. 

Until  recently,  the  KMO  billet  at  the  BCT  level  was  a  Functional  Area  (FA)  57 
0-3  (captain).  There  had  been  an  earlier  grade  plate  review  that  dropped  the  rank 
for  the  KMO  slot  down  from  an  0-4  (major).  Then,  the  FA  57  0-3  KMO  slot  was 
eliminated  from  the  BCT  Modified  Table  of  Organization  and  Equipment  (MTOE). 
More  recently,  the  Maneuver  Center  of  Excellence  (the  “owner”  of  the  BCT  MTOE 
and  the  entity  that  had  eliminated  the  0-3  KMO  slot)  identified  this  deletion  as  a 
“gap”  and  recommended  the  BCT  executive  officer  (XO)  as  the  KMO.  Based  on 
results  from  AWA  17.1,  assigning  BCT  KMO  responsibilities  solely  to  the  XO 
should  be  reconsidered.  Both  XO  and  KMO  can  be  full-time  roles.  The  BCT  XO 
might  be  assigned  overall  responsibility  for  the  unit’s  applied  KM  program,  but  he 
or  she  must  be  adequately  supported  in  perfonning  that  role. 

KM-related  research  indicates  that  viewing  KM  as  primarily  an  “IT  [information 
technology]  problem”  and  assigning  responsibility  for  KM  primarily  to  IT 
personnel  can  result  in  “unnecessary  IT  spending,  masses  of  unused  data,  and 
infonnation  overload”  for  consumers  of  that  information  (Vanini  and  Bochert 
2014).  IT  personnel  typically  do  not  know  what  information  is  relevant  for 
management  personnel.  That  same  literature  suggests  that  KM  is  primarily  a 
management  responsibility,  with  essential  technical  support  provided  by  the 
organization’s  IT  section.  These  results  support  the  finding  and  recommendation 
previously  provided.  The  lead  person  for  KM  in  a  tactical  unit  such  as  2/1  AD 
should  be  from  the  S-3  section.  However,  that  lead  must  be  supported  by  and  work 
closely  with  technical  personnel  from  the  S-6  (signal)  section. 
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In  the  interim  between  the  KMRC  and  the  start  of  AWA  17.1  (August-October 
2016),  the  test  unit’s  KMWG  did  not  meet  face  to  face.  Moreover,  only  2  of  the 
unit’s  designated  KMRs  signed  on  to  the  milSuite  virtual  collaboration  site 
provided  by  the  KM  Proponent.  The  idea  of  a  virtual  KMWG  collaboration  in  lieu 
of  face-to-face  working  group  contact  and  collaboration  did  not  work  as  planned 
prior  to  or  during  AWA  17.1.  Attempting  to  initiate  the  unit’s  KM  program  in 
parallel  with  the  run-up  to  an  intense  and  demanding  event  such  as  AWA  17.1 
resulted  in  too  little  time  and  key  personnel  availability  to  get  the  program  off  the 
ground  successfully.  As  a  consequence,  the  KM  analysis  team  had  very  little 
empirical  basis  on  which  to  address  the  applied  KM  training  program’s 
effectiveness  or  its  effect  on  KM  practices  or  mission  command  effectiveness 
during  the  subsequent  exercise. 

The  unit’s  KMO  later  remarked  that  a  milSuite  virtual  collaboration  site  might  be 
useful  if  it  was  used  in  combination  with  periodic  face-to-face  working  group 
meetings.  He  noted  that  the  virtual  collaboration  site  could  serve  as  a  forum  to 
exchange,  review,  and  critique  KM-related  products  such  as  KM  standard  operating 
procedures  (SOPs).  The  KMO  also  stated  that  during  program  initiation,  frequent 
face-to-face  working  group  meetings  might  be  essential  to  “gel”  the  KMWG  (i.e., 
create  an  integrated  functional  team)  and  “get  the  program  off  the  ground.”  Given 
that  the  KMWG  did  not  meet  face  to  face,  that  team  gelling  process  did  not  occur 
in  2/1  AD  prior  to  or  during  AWA  17.1. 

2.2  Assessment  Results 

Over  the  course  of  AWA  17.1,  SMEs  from  the  MCCoE,  in  collaboration  with 
supporting  ARL  team  personnel,  conducted  field  observations  and  guided 
interviews  with  unit  personnel  who  had  completed  the  KMRC  and  SLEO.  At  the 
end  of  the  exercise,  personnel  from  the  KM  Proponent’s  office  and  ARL  conducted 
focus  group  sessions  with  members  of  the  KMRC  and  several  of  the  test  brigade’s 
battalion  commanders  and  their  key  staff  members.  The  analysis  team  intended  to 
use  the  post-exercise  focus  group  sessions  to  collect  additional  information 
concerning  KM  training  effectiveness  and  the  program’s  perceived  impact  on 
mission  command  performance  during  the  preceding  AWA. 

Course  Effectiveness.  Going  into  the  applied  KM  demonstration  project,  the  ARL 
team  had  planned  to  use  what  is  tenned  the  Kirkpatrick  Model  for  assessing  KM 
training  effectiveness  (Kirkpatrick  and  Kirkpatrick  2006).  The  Kirkpatrick  Model 
considers  the  value  of  any  type  of  training  across  4  levels.  Level  1,  Reaction, 
evaluates  how  participants  respond  to  the  training:  Did  they  like  it?  Level  2, 
Learning,  measures  whether  trainees  actually  learned  the  material  in  question:  Do 
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they  think  (or  demonstrate  in  a  practical  exercise)  that  they  learned  the  course 
material?  Level  3,  Behavior,  considers  whether  the  trainees  are  able  to  apply  what 
they  learned  on  the  job:  Could  they  use  the  training  to  set  up  and  execute  an 
effective  applied  KM  program?  Level  4,  Results,  evaluates  whether  or  not  the 
training  positively  impacts  the  organization:  Does  the  unit  do  a  better  job  managing 
information  during  mission  command  operations? 

In  the  case  of  applied  KM  training  in  2/1  AD,  trainers  from  the  KM  Proponent’s 
office  administered  a  short  questionnaire  addressing  KMRC  trainees’  reaction  to 
the  training  following  course  completion.  Trainee  responses  concerning  how  the 
course  was  conducted  were  generally  positive.  Most  trainees  also  judged  that  they 
had  learned  the  course  material.  During  a  subsequent  interview  session,  the  brigade 
KMO  also  reported  that  he  liked  the  3 -week  KMQC  (Reaction)  and  asserted  that 
he  had  learned  the  material  in  question  (Learning).  That  said,  considerable  caution 
must  be  used  when  taking  trainees’  post- training  reaction  and  perceptions  regarding 
learning  at  face  value.  Prior  experience  with  Anny  training  similar  to  that  offered 
as  part  of  the  applied  KM  program  has  indicated  that  personnel  attending  new 
equipment-type  training  will  often  end  up  “knowing  about”  the  material  in  question 
but  may  not  actually  “know  how  to  do”  what  they  are  now  being  asked  to  do 
(Hawley  2007).  That  is,  course  participants  will  often  judge  the  training  favorably 
and  assert  that  they  learned  the  material.  However,  when  actually  called  upon  to 
apply  training-related  knowledge  and  skills  in  their  work  setting,  they  fall  short.  At 
that  initial  stage  of  program  familiarization  and  learning,  they  simply  “don’t  know 
what  they  don’t  know”.  The  training  evaluation  questionnaire  administered  at  the 
conclusion  of  the  KMRC  is  shown  as  Appendix  B.  The  response  range  was  from  1 
(Strongly  Disagree)  to  5  (Strongly  Agree).  The  midpoint  value  3  was  neutral: 
Neither  Disagree  nor  Disagree.  Means  and  medians  of  the  trainee  responses  to  each 
of  the  items  on  the  training  evaluation  questionnaire  are  provided  in  Table  1. 

Table  1  Means  and  medians  of  trainees  responses  to  items  on  the  KM  training  evaluation 
questionnaire  (n  =  15) 


Questionnaire  item _ Mean  Median 

1  now  have  a  better  understanding  of  what  KM  provides.  4.00  4 

I  better  understand  how  the  people,  process,  and  tool  components  of  KM  3.88  4 

support  the  organization. 

1  better  understand  how  knowledge  flows  to  create  shared  understanding.  4.06  4 

I  better  understand  how  KM  supports  the  operations  process.  4.13  4 

I  better  understand  how  to  apply  the  KM  process  within  my  staff  position  3.88  4 

and  role. 

I  better  understand  the  different  roles  and  responsibilities  of  people  3.81  4 

(Soldier,  KMO,  KMR,  chief  of  staff,  etc.)  in  the  KM  process. 
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Table  1  Means  and  medians  of  trainees  responses  to  items  on  the  KM  training  evaluation 
questionnaire  (n  =  15)  (continued) 


Questionnaire  item 

Mean 

Median 

1  better  understand  vertical  and  horizontal  collaboration. 

4.25 

4 

I  believe  this  training  will  enable  better  information  sharing  throughout  my 
unit. 

4.00 

4 

I  better  understand  how  to  leverage  tools  enabling  information  sharing. 

4.19 

4 

I  believe  I  can  use  the  knowledge  gained  in  training  to  assist  in  creating, 
executing,  and  evaluating  a  KM  unit  plan. 

4.31 

4 

I  believe  training  prepared  me  to  use  the  KM  maturity  model  to  conduct  a 
unit  KM  assessment. 

4.06 

4 

I  believe  the  KM  maturity  model  can  assist  me  in  identifying  knowledge 
sharing  weaknesses  in  my  unit. 

4.19 

4 

I  believe  my  leaders  will  embrace  KM  to  better  our  unit  shared 
understanding  and  collaboration. 

4.31 

4 

Because  the  applied  KM  program  was  not  fully  implemented  in  2/1  AD  prior  to  or 
during  AWA  1 7. 1 ,  it  was  not  possible  to  conduct  any  assessment  relevant  to  Levels 
3  and  4  of  the  Kirkpatrick  model.  Few  empirical  data  were  available  concerning 
whether  the  KMO  or  the  KMRs  were  able  to  apply  what  they  learned  during  the 
KM  training  sequence  on  the  job.  Similarly,  Level  4,  organizational  impact,  could 
not  be  validly  assessed. 

Maturity  Model  Utility.  Based  on  2  initial  applications,  the  KM  Maturity  Model 
showed  promise  as  an  effective  way  to  assess  the  status  of  KM  within  a  unit.  The 
initial  application  of  the  Maturity  Model  at  the  conclusion  of  the  KMRC  indicated 
that  2/1  AD’s  KM  Maturity  Level  was  rated  as  2.7.  A  second  application  of  the 
Maturity  Model  during  the  post-exercise  KM  focus  group  session  indicated  that  2/ 1 
AD’s  level  of  KM  maturity  had  improved  slightly  but  not  significantly:  it  was  still 
rated  as  between  2  and  3.  KM  Level  2  is  defined  as  “Some  KM  processes  are 
repeatable,  possibly  with  consistent  results.  Process  discipline  is  unlikely  to  be 
rigorous.”  Level  3  is  characterized  by  defined,  documented,  and  disciplined  KM 
processes  across  the  brigade  and  subordinate  tactical  units  that  are  also  subject  to 
some  degree  of  improvement  over  time. 

Based  on  observations  from  multiple  NIEs  and  the  current  AWA,  the  consensus 
opinion  of  ARL’s  KM  assessment  team  was  that  these  initial  assessments  of  2/1 
AD’s  KM  maturity  level  were  likely  overstated.  Based  on  the  definitions  cited, 
actual  KM  maturity  was  likely  closer  to  a  2  than  to  a  3.  Unit  observations  and  focus 
group  results  also  suggested  there  was  considerable  variability  in  KM  practices 
across  the  test  brigade’s  subordinate  battalions.  For  example,  one  battalion 
appeared  to  have  made  some  use  of  the  training  provided  in  the  KMRC,  while  other 
battalions  demonstrated  no  course  impact.  Some  of  this  variability  across 
subordinate  units  was  reasonably  attributable  to  the  fact  that  the  brigade  did  not 
have  a  disciplined  KM  framework.  A  reasonable  goal  going  forward  is  to  raise  the 
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brigade’s  overall  KM  maturity  level  to  an  actual  3.  This  would  require  that  the 
applied  KM  program  be  initiated  and  in  place  long  enough  to  demonstrate  positive 
results.  Merely  starting  the  program  is  not  sufficient.  KM  personnel  (KMO  and 
KMWG)  must  apply  what  they  have  learned  and  practice  KM -related  skills  in 
increasingly  challenging  mission  command  training  events.  The  Maturity  Model 
could  be  used  to  track  the  status  of  a  unit’s  KM  status  over  time  after  an  applied 
KM  program  has  been  introduced  and  program  execution  has  begun. 

ARL/HRED’s  analysis  team  also  noted  that  a  number  of  the  KM  Maturity  Model’s 
assessment  criteria  are  subjective,  which  can  lead  to  a  misrepresentation  of  a  unit’s 
actual  level  of  maturity.  A  common  criticism  of  Likert-style  self-evaluation 
questionnaires  is  that  raters  tend  to  overrate  themselves  (Babbitt  and  Nystrom 
1989).  That  is,  raters  tend  to  rate  themselves  as  being  better  than  they  actually  are. 
A  standard  method  for  reducing  subjectivity  in  Likert  scale  assessments  is  to 
provide  anchor  points  for  each  of  the  scale  values.  Scale  point  anchoring  would 
involve  providing  more  detailed,  situationally  relevant  descriptions  for  each  scale 
point  on  each  KM  criterion. 

A  second  observation  concerning  the  validity  of  the  KM  maturity  model  pertains 
to  the  relative  importance  of  the  various  groupings  of  rating  scales  used  in  the 
assessment.  Are  all  of  the  primary  KM  domains  used  in  the  model  equally  relevant 
to  actual  KM  practices  within  a  tactical  military  organization?  For  example,  overall 
KM  maturity  for  2/1  AD  appeared  to  be  overly  influenced  by  high  scale  scores 
within  the  Organization/Culture  domain.  The  scales  defining  the 
Organization/Culture  domain  are  important  with  respect  to  how  well  an 
organization  is  able  to  execute  a  functional  KM  program.  However,  it  is  possible  to 
have  a  “good”  organization,  as  rated  on  those  scales,  that  does  not  manage  its 
infonnation  well.  That  said,  the  maturity  rating  process  within  a  tactical 
organization  should  focus  more  on  a  unit’s  actual  information  management  (IM) 
practices  and  less  on  supporting  organizational  characteristics.  The  importance 
assigned  to  the  various  maturity  domains  should  thus  be  adjusted  to  better  focus  on 
and  reflect  actual  IM  practices  as  applied  within  a  tactical  military  organization. 
Weighting  the  KM  domains  to  better  focus  on  a  unit’s  actual  IM  practices  is  another 
potential  way  to  improve  the  Maturity  Model’s  validity  and  utility. 

The  Army  KM  Proponent’s  applied  KM  program  is  evolving.  The  KM  Proponent’s 
initial  concept  was  that  a  training  program  consisting  of  the  KMQC,  KMRC,  and 
SLEO  along  with  the  virtual  milSuite  collaboration  site  and  occasional  unit  contact 
would  be  sufficient  to  get  an  effective  applied  KM  program  up  and  running  in  a 
target  unit.  Experience  from  organizational  development  initiatives  similar  to  KM 
suggests,  however,  that  training  alone  is  not  sufficient  to  initiate  an  effective 
intervention  in  a  target  organization  (Schein  2004).  As  noted  previously,  unit 
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personnel  attending  such  training  will  often  end  up  knowing  about  the  program  in 
question,  but  will  not  actually  know  how  to  do  what  they  are  now  being  asked  to 
do.  Some  amount  of  follow-on  consultation  and  guided  practice  is  necessary  to 
move  from  knowing  about  to  knowing  how  to  do,  to  actually  doing  those  things 
successfully  in  their  job  context. 

In  the  case  of  the  Army’s  KM  program,  the  KM  Proponent  does  not  have  sufficient 
resources  to  support  both  training  and  extensive  follow-on  consultation  across  all 
of  the  Army’s  BCTs;  hence  their  effort  to  initiate  a  milSuite  virtual  collaboration 
site  that  might  take  the  place  of  continuing  face-to-face  contact  with  receiving 
organizations.  Experience  with  initiatives  similar  to  KM  also  suggests  that  beyond 
consultation  and  other  forms  of  reach-back,  it  is  useful  to  provide  receiving 
organizations  with  prepackaged  model  solutions  (i.e.,  “school”  solutions)  that  the 
receiving  organization  can  use  to  quickly  get  their  applied  program  started.  These 
prepackaged  solution  sets  can  be  customized  to  fit  the  target  unit’s  circumstances. 
They  also  enable  a  program  “quick  start”  (a  necessary  attribute  to  enable  initial 
program  “successes”  and  sustain  organization  interest),  and  explicitly  demonstrate 
to  the  organization  “what  right  looks  like”  in  critical  program  areas.  Follow-on 
consultation  and  guided  practice  in  “doing”  KM  (also  essential)  are  then  organized 
around  these  prepackaged  solutions  sets.  Contact  with  the  receiving  organization 
thus  has  a  concrete,  action-oriented  focus  on  supporting  incremental  KM 
improvements  in  the  target  organization. 

3.  Conclusion:  Toward  an  Effective  Approach  to  Enhancing  KM 
Proficiency  in  Tactical  Units 

As  stated  previously,  the  overall  objective  of  the  applied  KM  demonstration  project 
in  2/1  AD  during  AWA  17.1  was  two-fold.  The  first  objective  was  to  assess  the 
effectiveness  of  the  Army  KM  Proponent’s  3  doctrine-based  KM  courses.  The 
second  objective  was  to  assess  the  KM  Maturity  Model’s  utility  as  a  tool  for 
commanders  to  determine  KM  maturity  within  their  unit  and  to  lay  out  concrete 
paths  for  KM  improvements.  These  capstone  objectives  were  further  broken  down 
into  5  specific  assessment  questions.  Demonstration  project  results  as  they  apply  to 
these  assessment  questions  are  summarized  in  the  following  section. 
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3.1  Demonstration  Project  Results  in  Relation  to  Subordinate 
Assessment  Questions 

Assessment  Question  1:  How  effective  is  the  KMQC  in  preparing  KMOs  to 
execute  the  duties  and  responsibilities  of  a  KM  professional? 

The  2/1  AD’s  designated  KMO  attended  the  KMQC  prior  to  the  start  of  AWA  17.1. 
The  KMO  reported  favorably  on  the  course  and  indicated  that  he  had  learned  the 
course  material  and  was  prepared  to  serve  in  the  KMO  role.  That  said,  the 
assessment  team  was  not  able  to  obtain  any  empirical  data  pertaining  to  the 
KMQC’s  actual  training  effectiveness  or  later  impact  on  the  unit’s  mission 
command  effectiveness.  The  applied  KM  program  in  2/1  AD  was  not  implemented 
as  initially  planned.  The  KMO  later  noted  that  as  a  CW2  Network  Services 
Technician  he  was  not  the  right  person  to  serve  as  KMO.  He  judged  that  he  did  not 
have  a  sufficient  background  in  battle-staff  mission  command  operations  to 
function  effectively  in  the  KMO  role.  The  KMQC  by  itself  cannot  be  expected  to 
compensate  for  a  lack  of  relevant  staff-related  training  and  experience. 

Assessment  Question  2:  How  effective  was  the  KMRC  in  preparing  KMRs  to 
support  the  KMO  in  conducting  unit  KM  operations? 

As  with  the  situation  involving  the  KMO,  KMRs  attending  the  KMRC  had  a 
generally  favorable  view  of  the  course  and  reported  that  they  had  acquired  the 
necessary  knowledge  and  skills.  However,  since  the  KM  program  in  2/1  AD  did 
not  get  off  the  ground  as  initially  planned,  there  was  no  evidence  to  support  the 
course  attendee’s  assertions  that  they  learned  the  course  material  or  would  have 
been  able  to  apply  what  they  learned  on  the  job.  The  KMWG  did  not  meet  face  to 
face,  and  KMR  participation  with  the  milSuite  virtual  collaboration  site  was 
minimal.  Hence,  there  was  no  empirical  evidence  that  the  unit’s  KMRs  were 
prepared  to  support  the  KMO  in  structuring  and  conducting  applied  KM  operations. 
The  only  positive  results  pertaining  to  this  assessment  question  were  observed  in 
one  of  the  brigade’s  subordinate  battalions.  That  battalion  commander  used  his 
trained  KMRs  to  staff  his  command  vehicle  and  support  enhanced  IM  within  his 
unit.  Once  again,  however,  these  conclusions  are  observational  and  anecdotal. 

Assessment  Question  3:  How  effective  is  the  SLEO  in  ensuring  senior  leaders 
understand  KM  fundamentals  and  how  it  enables  mission  command? 

With  the  exception  of  the  observation  concerning  the  battalion  commander 
referenced  in  Question  2,  there  was  no  empirical  evidence  bearing  on  this 
assessment  question.  Only  2  battalion-level  commanders  attended  the  post-exercise 
KM  focus  group  session.  No  personnel  from  the  brigade’s  command  group  or 
senior  staff  attended  the  post-event  KM  focus  group  session. 
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The  ARL  personnel  on  the  KM  assessment  team  observed  that  the  test  unit’s 
command  and  senior  staff  personnel  did  not  appear  particularly  interested  in  the 
applied  KM  demonstration  project.  They  did  not  interact  much  with  the  presenters 
during  the  SLEO  (remaining  mostly  passive),  and  attendance  at  the  post-exercise 
KM  focus  group  session  was  sparse.  In  fact,  the  post-exercise  focus  group  session 
was  late  getting  started  because  designated  attendees  did  not  appear.  Other  unit 
activities  had  been  scheduled  over  the  focus  groups  session  in  spite  of  the  fact  that 
the  KM  focus  groups  session  was  scheduled  well  in  advance  of  the  designated  time. 
ARL  assessment  team  members  debated  among  themselves  concerning  the  reason 
for  this  seeming  lack  of  interest  in  the  applied  KM  demonstration  project.  One 
experienced  member  of  the  ARL  team  opined,  “They  appear  to  think  they  are  OK, 
and  don’t  need  this  program.”  That  is,  the  unit  appeared  to  be  complacent  with 
respect  to  KM  and  satisfied  with  the  status  quo.  This  observation  is  similar  to 
observations  pertaining  to  other  aspects  of  battle  staff  training  and  staff 
perfonnance  deficiencies  reported  in  the  literature  (Thompson  et  al.  1991;  Howse 
and  Cross  1999). 

Thompson  et  al.’s  field  studies  of  battle  staff  training  and  performance  revealed 
systemic  training  problems  in  preparing  officers  to  assume  battalion  staff  duties. 
These  authors  reported  that  most  training  received  by  new  staff  officers  was  ad  hoc 
on-the-job  training.  The  ARL  team’s  observations  across  NIEs  indicated  that  this 
situation  has  not  changed  (Hawley  2015;  Hawley  and  Swehla  2016).  Moreover, 
Thompson  et  al.’s  interviews  with  battalion  commanders  and  senior  staff  officers 
further  indicated  that  unit  leaders  did  not  think  that  leader  and  staff  training 
exercises  (e.g.,  Command  Post  Exercises)  were  important  and  did  not  think  it  was 
necessary  to  conduct  these  exercises  frequently.  These  authors  concluded  that 
battalion  commanders  and  senior  staff  officers  “lacked  the  perception”  necessary 
to  assess  their  true  condition  because  “they  had  not  had  sufficient  training  and 
experience  to  know  what  they  must  do  to  operate  effectively”  (Thompson  et  al., 
p.  32).  Mission  command  problems  began  to  emerge  only  under  the  stressful 
combat-like  conditions  encountered  at  the  CTCs,  where  units  must  truly  plan, 
prepare,  and  execute  using  mission  command  resources  as  they  would  under  actual 
combat  conditions.  These  requirements  include  staff  actions  and  synchronization 
of  command  and  staff  functions  under  harsh  time  and  resource  constraints. 
Thompson  et  al.  concluded  that  CTCs  may  be  the  only  place  where  commanders 
and  staff  are  sufficiently  taxed  to  reveal  flaws  in  mission  command  training.  Such 
flaws  are  not  revealed  during  routine  home  station  training  or  less-stressful  and 
dynamic  exercises.  Recent  reports  from  the  CTCs  suggest  this  is  still  the  case 
(MCNS  2017). 
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Recent  literature  sources  also  suggest  that  the  emphasis  on  counterinsurgency 
(COIN)  and  stability  operations  for  the  past  15  years  has  resulted  in  a  set  of 
command  and  staff  practices  suitable  for  COIN  but  less  suitable  for  traditional 
maneuver  warfare.  Moreover,  commanders  and  senior  staff  may  not  be  aware  that 
practices  that  proved  adequate  for  COIN  and  stability  operations  may  not  be 
sufficient  for  mission  command  during  high-intensity  maneuver  operations 
(MCCTS  2016).  With  respect  to  knowledge  and  information  management 
(KM/IM),  that  same  report  noted  that  the  “SOPs  and  TTPs  developed  to  maintain 
digital/analog  balance  while  conducting  unit  KM  and  IM  processes  [during  COIN 
and  stability  operations]  may  not  be  directly  applicable  to  future  decisive 
action/unified  land  ops  [operations]  needs”  (MCCTS  2016,  p.  16). 

Assessment  Question  4:  Does  the  KM  Maturity  Model  enable  the  unit  to  fully 
assess  unit  KM  posture? 

As  noted  previously,  the  KM  Maturity  Model  shows  promise  as  a  tool  for  assessing 
a  unit’s  KM  posture.  Demonstration  results  suggested  that  the  model  could  benefit 
from  being  more  objective  with  respect  to  a  unit’s  actual  KM  and  IM  proficiency. 
Suggestions  for  making  the  model  more  objective  and  indicative  of  a  unit’s  actual 
KM  and  IM  proficiency  were  provided  in  Section  2.2  of  this  report.  The  model’s 
utility  could  also  be  improved  by  providing  users  with  concrete,  action-oriented 
steps  for  improving  their  KM  posture,  such  as  moving  from  Level  2  to  Level  3  or 
from  Level  3  to  Level  4. 

Assessment  Question  5:  How  does  fonnalized  training  contribute  to  shared 
understanding  throughout  the  operations  cycle? 

Because  of  the  failure  to  implement  an  applied  KM  program  in  2/1  AD  prior  to 
AWA  17.1,  there  were  no  results  that  pertain  directly  to  this  assessment  question. 
This  question  relates  to  Level  4  of  the  Kirkpatrick  model,  and  no  results  relating  to 
the  program’s  effect  on  mission  command  or  unit  performance  were  obtainable 
during  AWA  17.1. 

3.2  Observations  and  Recommendations 

The  KM  assessment  results  discussed  in  the  previous  section  clearly  demonstrated 
that  getting  a  program  such  as  applied  KM  up  and  running  in  a  tactical  unit  is  a 
challenging  undertaking.  Considerable  program  structuring,  training, 
developmental  work,  and  command  emphasis  on  the  part  of  the  target  unit  are 
required  to  get  such  a  program  off  the  ground  and  functioning  successfully. 
Maintaining  an  effective  KM  program  in  the  face  of  routine  personnel  turnover  is 
yet  another  challenge.  At  the  conclusion  of  the  KMRC  in  August  2016,  the  structure 
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and  initial  personnel  fill  for  an  applied  KM  program  in  2/1  AD  were  in  place. 
However,  there  was  not  sufficient  time  or  key  personnel  availability  to  turn  that 
program  structure  and  concept  into  effective  program  execution  prior  to  or  during 
AWA  17.1.  Consequently,  the  KM  assessment  team  had  very  little  empirical  data 
with  which  to  address  any  of  the  assessment  criteria  addressing  KM  training 
effectiveness  or  the  program’s  operational  effect.  In  spite  of  these  limitations,  the 
assessment  did  provide  several  observations  and  recommendations  concerning  KM 
program  effectiveness  and  potential  improvements  going  forward.  These 
observations  and  recommendations  are  summarized  in  the  following  paragraphs. 

Observation:  The  KMRC  primarily  informs  Soldiers  about  KM  principles  and 
concepts.  Although  the  course  output  does  result  in  initial  products  for  the  KMWG, 
additional  hands-on,  practical  emphasis  during  training  is  necessary. 

Recommendation:  The  KM  Proponent  should  coordinate  with  operating  and 
generating  force  KMOs  to  customize  the  KMRC  to  reflect  the  unit’s  KM  program. 
Another  suggestion  is  to  include  the  unit’s  KMO  as  a  key  facilitator  during  the 
KMRC. 

Observation:  The  KM  Maturity  Model  shows  promise  as  an  effective  way  to  assess 
the  status  of  KM  within  a  tactical  unit. 

Recommendation:  The  MCCoE  and  KM  Proponent  should  investigate  the  use  of 
the  Maturity  Model  to  track  the  status  of  a  unit’s  KM  over  time  after  an  applied 
KM  program  has  been  introduced  and  program  execution  has  begun.  Data  from 
such  repeated  applications  could  also  serve  to  refine  the  model  and  better  define  in 
operational  terms  what  is  meant  by  a  unit’s  KM  “maturity”. 

Observation:  The  KMRC  and  the  SLEO  were  catalysts  in  one  of  the  brigade’s 
battalion-level  units  to  mature  and  enhance  its  KM  program.  The  unit  commander 
appeared  to  have  a  high  appreciation  for  information  flow  and  KM;  therefore,  he 
placed  Soldiers  in  key  positions  (battle  captain,  battle  NCO,  and  commander’s 
driver)  to  attend  the  KMRC.  These  Soldiers,  anned  with  a  new  understanding  of 
KM,  were  allowed  to  implement  improvements  to  the  unit’s  KM  program, 
information  flow,  and  information  display  at  the  battalion  level. 

Recommendation:  This  observation  is  an  example  of  a  KM  best  practice.  It  also 
illustrates  the  potential  benefits  of  enhanced  KM  practices  in  a  tactical  unit. 
Examples  such  as  this  should  be  used  to  refine  the  SLEO  “sales  pitch”  and  provide 
practical  “how  to”  advice  to  units  initiating  an  applied  KM  program. 

Observation:  The  BCT’s  KMO  was  not  familiar  with  staff  operations  and  was  too 
busy  as  a  Network  Services  Technician  to  initiate  the  applied  KM  program  in 
conjunction  with  the  KMWG.  The  2/1  AD  KMO  (a  CW2  Network  Services 
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Technician  holding  MOS  255A)  was  not  the  right  person  to  fill  that  role  at  the 
brigade  level.  A  person  from  the  unit’s  S-3  section  with  a  background  in  the 
Military  Decision  Making  Process  and  battle  staff  operations  would  have  been  a 
more  suitable  choice. 

Recommendation:  The  KMO  at  the  brigade  level  should  be  key  member  of  the 
S-3  staff  section.  Moreover,  the  MCCoE  should  investigate  assigning  battle 
captains  or  NCOs  as  KMRs  in  subordinate  units.  The  MCCoE  also  should  explore 
the  need  for  a  mission  command  integration  specialist  (e.g.,  a  Mission  Command 
Digital  Master  Gunner)  to  be  assigned  to  the  commander’s  vehicle  to  support 
enhanced  IM. 

Observation:  KMWG  virtual  collaboration  using  the  KM  proponent’s  milSuite  site 
did  not  work  out  as  intended. 

Recommendation:  A  milSuite  virtual  collaboration  site  might  be  useful  in 
combination  with  periodic  face-to-face  KMWG  meetings.  Face-to-face 
collaboration  by  the  KMWG  is  essential  during  applied  KM  program  start-up. 

Observation:  As  emphasized,  training  alone  is  not  sufficient  to  initiate  an  effective 
applied  KM  program  in  a  tactical  unit.  Some  form  of  follow-on  consultation  with, 
and  sample  product  support  to,  the  receiving  organization  is  necessary  for 
successful  program  start-up.  Solid  command  support  also  is  essential  to  getting  an 
effective  applied  KM  program  up  and  running.  For  example,  during  an  interview 
session  following  the  KMRC,  one  of  the  KMRs  was  questioned  about  the  potential 
success  of  the  program.  He  noted  that  “If  the  commander  wants  it  to  happen,  it  will 
happen.  If  the  commander  has  other  priorities,  it  probably  won’t  take  hold”. 

Recommendation:  The  MCCoE  and  KM  Proponent  should  further  pursue  the 
implementation  of  prepackaged  KM  model  solutions  (i.e.,  “school”  solutions)  that 
a  receiving  organization  can  use  to  quickly  get  the  program  started.  Solutions 
should  include  templates  (SharePoint,  Microsoft  Office,  Command  Post  of  the 
Future  Pasteboards,  etc.),  SOP  examples,  and  sample  TTPs.  These  products  along 
with  follow-on  consultation  will  enable  a  receiving  unit  to  customize  the  program 
to  fit  organizational  circumstances  and  capabilities.  Such  templates  could  give  the 
receiving  organization  a  concrete  stating  point  from  which  to  tailor  their  own 
applied  KM  program. 

Observation:  Units  tend  to  have  a  separate  SOP  or  SOP  Annex  for  KM  that 
describes  their  processes  and  procedures  for  IM. 

Recommendation:  The  KM  Proponent  should  express  in  doctrine  that  infonnation 
flows,  infonnation  structure,  infonnation  products,  and  information  displays  be 
incorporated  into  the  main  body  of  the  unit  SOP  or  in  the  applicable  functional 
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areas  of  the  SOP.  The  KM  SOP  or  SOP  Annex  should  contain  only  the 
administrative  details  of  the  unit’s  KM  program. 

Observation:  A  number  of  the  KM  Maturity  Model’s  assessment  criteria  are 
subjective,  which  can  lead  to  a  misrepresentation  of  a  unit’s  actual  level  of  KM 
maturity  and  uncertainty  regarding  specific  directions  for  KM  improvements. 

Recommendation:  The  KM  proponent  should  develop  objective  measures  of  the 
maturity  of  a  unit’s  KM  program.  These  objective  measures  should  include  the 
number  of  KMOs  and  KMRs  along  with  their  recommended  positions  and  level  of 
prior  training  and  experience.  Such  objective  measures  would  provide  a  more 
accurate  assessment  of  a  unit’s  KM  maturity.  Equipped  with  a  more  objective 
Maturity  Model,  the  KM  proponent  could  then  develop  prioritized,  action-focused 
paths  for  improving  a  unit’s  KM  proficiency.  These  paths  should  be  formulated  to 
lay  out  the  incremental  steps  necessary  to  move  up  the  KM  maturity  scale  (from 
Level  1  to  2,  from  Level  2  to  3,  etc.). 

Observation:  The  2/1  AD  experienced  challenges  disseminating  information  and 
managing  knowledge  with  coalition  partners.  Currently,  coalition  KM  integration 
is  not  part  of  the  curriculum  of  any  of  the  Anny’s  KM  courses  or  as  part  of  the 
Mission  Command  Digital  Master  Gunner  course. 

Recommendation:  Army  organizations  responsible  for  developing  coalition 
interoperability  standards  (ABCA  [American,  British,  Canadian,  Australian,  and 
New  Zealand  Annies]  Program;  North  Atlantic  Treaty  Organization  [NATO]; 
Multilateral  Interoperability  Program  [MIP];  etc.)  should  develop  course  modules 
for  incorporation  into  existing  Army  KM  courses. 

Observation:  ABCA  has  not  developed  TTPs  for  integrating  coalition  partners  into 
operations.  Current  doctrine  and  standards  are  primarily  conceptual  ideas  and 
general  guidance.  Soldiers  need  supporting  concrete  examples,  check  lists, 
templates,  and  TTPs  to  apply  this  conceptual  material. 

Recommendation.  Anny  organizations  responsible  for  developing  coalition 
interoperability  standards  (ABCA,  NATO,  MIP,  etc.)  should  develop  TTPs 
providing  detailed  instructions  for  the  Anny’s  implementation  of  coalition 
interoperability  standards.  Soldiers  should  have  reference  manuals  that  describe 
various  ways  of  moving  and  displaying  known  information  exchange  requirements 
with  known  (i.e.,  ABCA  and  NATO)  coalition  partners. 

Observation:  Anny  tactical  units  do  not  often  train  with  coalition  forces. 
Therefore,  they  lack  practical  experience  integrating  with  coalition  forces. 
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Recommendation.  When  practical,  Anny  training  events  should  include  coalition 
partners.  It  is  unreasonable  to  expect  that  effective  coalition  KM  integration  will  be 
obtained  without  vetted  procedures  and  extensive  practice  in  applying  those 
procedures. 

3.3  Discussion 

So  how  should  the  issue  of  enhancing  KM  proficiency  in  tactical  units  be 
approached  going  forward?  Beyond  the  results  and  discussion  presented 
previously,  there  are  several  additional  observations  and  lessons  from  the  applied 
KM  demonstration  project  in  2/1  AD  during  AWA  17.1  that  have  some  relevance 
to  this  question. 

First,  the  demonstration  project  in  2/1  AD  during  AWA  17.1  did  not  provide  a 
suitable  basis  for  an  assessment  of  the  KM  Proponent’s  approach  to  enhancing  KM 
proficiency  in  tactical  units.  The  demonstration  project  provided  sparse  evidence 
concerning  the  effectiveness  of  any  of  the  KM-related  courses,  the  utility  of  the 
KMWG  concept,  or  the  program’s  impact  on  mission  command  or  unit 
perfonnance. 

That  said,  the  difficulty  of  implementing  a  successful  applied  KM  program  in  a 
tactical  unit  such  as  2/1  AD  should  not  be  underestimated.  Considerable  time, 
preliminary  work,  and  sustained  effort  on  the  part  of  the  unit — along  with  a  high 
level  of  command  involvement  and  support — are  necessary  for  an  applied  KM 
program  to  be  successful.  Attempting  to  get  an  intervention,  such  as  applied  KM 
implemented  in  parallel  with  the  run-up  to  an  intense  and  demanding  event  such  as 
an  AWA,  meant  that  the  unit  could  not  devote  sufficient  time  or  personnel  resources 
to  get  the  program  up  and  running  within  the  time  and  resources  allocated.  Future 
attempts  to  assess  KM  program  effectiveness  should  be  made  in  a  tactical  unit 
willing  and  able  to  devote  the  time  and  resources  necessary  for  a  fair  and  valid 
assessment  of  the  program’s  merits  and  to  suggest  potential  areas  for  improvement. 

A  second  caution  regarding  applied  KM  programs  is  that  research  results  from 
examining  KM  implementation  in  private-sector  organizations  are  not  positive,  for 
example  Vanini  and  Bochert  (2014)  reported  that  only  1/6  of  all  private-sector  KM 
initiatives  were  judged  to  contribute  significantly  to  corporate  success  within  the 
first  2  years  after  their  implementation.  Consequently,  the  majority  of  private-sector 
KM  initiatives  eventually  are  abandoned  as  “unproductive”.  Experience  outside  the 
Anny  also  has  indicated  that  applied  KM  programs  are  difficult  to  sustain, 
particularly  in  organizations  having  high  personnel  turnover.  This  latter  point  is 
significant  because  Anny  tactical  units  in  general  and  battle  staffs  in  particular  have 
high  levels  of  personnel  turnover.  For  example,  Thompson  et  al.  (1991)  reported 
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that  the  average  tenure  of  battle  staff  members  in  the  units  they  studied  was  4.6 
months.  These  results  suggest  that  the  potential  for  KM  program  “success”  is  not 
substantially  different  from  other  organizational  interventions  such  as  Total  Quality 
Management  or  Lean  Six  Sigma.  For  a  variety  of  reasons,  most  organizations 
struggle  to  make  such  interventions  pay  off  (Schein  2004). 

Results  from  across  a  number  of  venues  suggest  that  IM  proficiency  (the  crux  of  an 
applied  KM  program)  is  an  important  contributor  to  mission  command 
effectiveness  (Howse  and  Cross  1999;  MCTPTDAWE  2014).  This  is  particularly 
true  for  an  increasingly  information-intensive  setting  such  as  network-enabled 
mission  command.  These  same  results  also  indicate  that  Army  tactical  units 
typically  do  not  do  an  adequate  job  of  IM,  and  that  IM  “gap”  affects  mission 
command  and  unit  performance.  Hawley  and  Swehla’s  (2016)  observations  across 
multiple  NIEs  support  these  earlier  results.  The  KM  Proponent’s  current  approach 
to  implementing  applied  KM  in  tactical  units  represents  one  potential  way  to 
enhance  the  IM  proficiency  of  tactical  units.  However,  as  results  from  the  applied 
KM  demonstration  effort  during  AWA  17.1  indicated,  focused  and  sustained  unit 
effort  after  initial  program  implementation  are  critical  to  program  success. 

In  an  experimental  study  of  IM  practices  in  operations  centers,  Howse  and  Cross 
(1999)  reported  results  suggesting  that  the  best  way  to  improve  IM  performance  in 
digitized  operations  centers  was  to  incorporate  IM  skills  directly  into  the  routine 
training  provided  to  battle  staff  members.  Specific  areas  of  training  emphasis 
mentioned  in  this  regard  included  1)  distribution  of  IM  activities  and 
responsibilities  across  members  of  the  battle  staff,  2)  procedures  for  screening 
incoming  infonnation,  and  3)  proactive  anticipation  of  information  needs.  Beyond 
these  staff-focused  skill  development  areas,  these  authors’  results  also  stressed  the 
need  to  specify  1)  the  manner  in  which  each  digital  subsystem  of  the  digitized 
mission  command  suite  should  be  used  in  IM,  2)  the  unique  capabilities  of  each 
subsystem  to  support  IM,  and  3)  efficient  transfer  of  data  across  mission  command 
subsystems. 

Howse  and  Cross’s  conclusions  are  similar  to  those  provided  initially  in  Hawley 
(2015)  and  followed  up  in  Hawley  and  Swehla  (2016).  Hawley  (2015)  described 
and  discussed  the  effects  of  KM/IM  “breakdowns”  in  NIE  CPs  that  contributed  to 
battle-staff  sensemaking  deficiencies.  Sensemaking  is  defined  as  a  motivated, 
continuous  effort  to  understand  connections  to  understand  their  trajectory  and  act 
effectively  (Klein  et  al.  2006).  Sensemaking  is  the  foundation  of  situation 
awareness,  a  key  enabler  of  mission  command  effectiveness.  Hawley  and  Swehla 
(2016)  reported  results  indicating  that  the  battle  staffs  observed  during  the  NIEs 
were  not  sufficiently  trained  (individually  or  collectively)  or  experienced  enough 
to  achieve  the  level  of  staff  integration  essential  for  effective  IM.  The  O/C  Drop 
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Card  cited  previously  indicated,  for  example,  that  the  brigade  was  “piecemealing” 
the  fight.  That  is,  each  mission  command  staff  component  within  the  CP  was 
operating  semi-autonomously  with  little  overall  coordination  by  the  battle  staff. 
Similar  observation  regarding  synchronization  of  battle  staff  activities  has  also  been 
reported  in  other  sources  (see,  for  example,  MCTPTDAWE  2014).  Effective  battle 
staff  integration  is  an  essential  prerequisite  for  effective  IM,  staff  synchronization, 
and  battle  tracking  (Sauer  1996). 

Hawley  and  Swehla  (2016)  also  suggested  that  IM,  and  to  some  extent  applied  KM, 
might  best  be  approached  as  an  aspect  of  what  they  tenned  operational  integration. 
Recall  from  the  Introduction  that  operational  integration  refers  to  incorporating  new 
materiel  into  CP  and  mission  command  processes  and  procedures  as  well  as 
adjusting  existing  processes  and  procedures  to  reflect  the  capabilities  of  new 
systems  and  technologies.  The  central  idea  is  to  emphasize  means  for  the  effective 
use  of  new  materiel  to  efficiently  manage  the  wealth  of  data  provided  by  those 
capabilities.  It  has  been  noted  that  new  technology  often  changes  the  nature  of  the 
work  that  technology  is  intended  to  support.  This  statement  also  applies  to  IM 
practices  in  modernizing  CPs.  New  systems  and  capabilities  will  change  how 
tactical  units  can  and  should  approach  KM/IM  in  their  CPs. 

As  noted,  another  important  aspect  of  KM  in  tactical  units  involves  maintaining  a 
core  of  unit  expertise  over  time  as  personnel  arrive  at  and  depart  the  unit.  This 
represents  an  important  aspect  of  KM-related  perfonnance  management  at  the 
intersection  of  unit  training  and  local  personnel  management.  The  glue  that  holds 
systems  of  systems  such  as  a  CP  together  and  makes  them  more  than  a  collection 
of  hardware  is  Soldier  expertise.  However,  there  are  high  rates  of  personnel 
turnover  in  all  military  organizations.  The  human  parts  keep  passing  through  the 
system,  so  to  speak.  Thus,  even  though  a  unit  is  combat-ready  one  day,  it  may  not 
be  combat-ready  the  next  day  unless  the  expertise  of  the  personnel  departing  is 
continually  replaced  by  the  newly  acquired  skills  of  those  who  have  recently 
arrived.  This  high  turnover  rate  of  personnel  and  the  need  for  continual 
replenishment  of  expertise  is  an  important  consideration  in  unit  training  planning 
and  local  personnel  management. 

Maintaining  essential  levels  of  unit  expertise  in  critical  areas  such  as  KM  cannot  be 
left  solely  to  the  whims  of  the  Army’s  formal  personnel  assignment  system.  Units 
must  be  proactive  in  managing  available  KM  expertise,  as  with  any  other  critical 
resource.  During  discussions  of  this  topic  with  brigade  and  battalion  staff  members, 
they  remarked  that  maintaining  a  consistent  level  of  expertise  in  key  functional 
areas  is  a  difficult  challenge  for  a  unit’s  leadership.  Loss  of  a  few  key  specialists 
often  has  a  significant  impact  on  the  unit’s  performance  capabilities — single  points 
of  personnel-related  performance  risk.  That  said,  it  is  important  for  commanders 
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and  senior  staff  to  carefully  plan  for  staffing  the  unit’s  KM  function  in  terms  of 
battle  rostering,  longevity,  training,  priority  back  fill,  and  alternate  personnel. 

Enhancing  and  maintaining  KM/IM  proficiency  in  network-enabled  CPs  is  a  tough 
challenge.  Extensive  training,  guidance,  product  support,  battle  staff  continuity,  and 
relevant  practice  by  the  battle  staff  working  as  a  team  are  required  for  effective 
performance.  That  being  the  case,  increased  use  of  infonnation  automation  is  often 
suggested  as  a  solution  to  information  overload  and  KM/IM  problems  in 
modernizing  CPs  (see,  for  example,  MCCTS  2016).  Infonnation  automation  refers 
to  automating  data-  and  infonnation-handling  activities  in  fast-paced  information- 
rich  settings  such  as  a  network-enabled  CP  (Billings  1996).  The  idea  behind  the 
push  to  use  automation  to  relieve  the  battle  staff  of  the  demands  associated  with 
KM/IM  can  be  summarized  as  follows:  “Just  automate  the  information  handling 
requirements  in  modernizing  CPs,  and  we’ll  be  relieved  of  most  if  not  all  the 
training  and  human  perfonnance  demands  associated  with  effective  KM  or  IM. 
These  new  systems  will  take  care  of  that  for  us”.  Under  such  an  approach,  various 
software  applications  would  be  used  to  assist  (or  replace)  the  battle  staff  in 
perfonning  IM  processes.  Infonnation  automation  support  might  take  the  form  of 
a  standalone  support  system  refened  to  as  a  Commander’s  Assistant  or 
Commander’s  Toolkit  (or  something  similar),  or  might  be  embedded  in  the 
software  of  an  existing  mission  command  system. 

Increased  use  of  information  automation  has  the  potential  to  be  beneficial  in  many 
ways  and  enable  CP  operations  that  otherwise  might  be  difficult  to  support. 
However,  caution  is  suggested  because  infonnation  automation  can  have  negative 
side  effects  if  not  implemented  properly.  For  example,  a  recent  Federal  Aviation 
Administration  (FAA)  report  noted  that  information  automation  improperly  applied 
in  automated  flight  management  systems  could  increase  pilot  workload,  increase 
head-down  time,  distract  the  flight  crew  from  higher  priority  tasks,  and  contribute 
to  crew  communication  and  coordination  issues  (FAA  2013).  Moreover,  IM 
vulnerabilities  can  occur,  especially  if  users  are  not  aware  of  assumptions  made  in 
the  support  system’s  design  or  if  the  infonnation  presented  to  users  is  not  fully 
understood.  Used  in  this  context,  the  tenn  “vulnerability”  refers  to  characteristics 
or  issues  that  render  a  system  or  process  more  likely  to  break  down  or  fail  when 
faced  with  unusual  or  ambiguous  situations.  This  downside  of  automated  system 
perfonnance  is  often  referred  to  as  the  “brittleness  problem  of  automata”  (Hollnagel 
and  Woods  2005). 

These  cautions  from  the  flight  management  arena  are  relevant  to  providing 
automated  support  for  the  KM/IM  activities  that  take  place  within  contemporary 
CPs.  The  idea  that  information  automation  can  be  used  as  a  simple  replacement  for 
Soldier  KM/IM  responsibilities  exemplifies  one  of  the  “deadly  myths”  associated 
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with  automated  systems  (Bradshaw  et  al.  2013).  Another  persistent  myth  associated 
with  automation  is  that  its  use  will  lessen  user  training  requirements.  A  wealth  of 
human  factors  research  and  operational  experience  indicates  that  such  is  not  always 
the  case.  The  more  usual  result  is  that  automation  increases  training  requirements 
to  match  the  added  complexity  often  associated  with  automated  systems  (see,  for 
example,  Bainbridge  1983). 

Automation  can  be  used  to  supplement  and  extend  human  capabilities  but  not 
substitute  for  them  (see,  for  example,  Wickens  et  al.  2013).  But  even  then, 
providing  automated  support  to  a  function  such  as  KM/IM  must  be  approached 
cautiously  and  with  a  solid  understanding  of  how  a  proficient  battle  staff  uses  (or 
might  use)  incoming  data  to  generate  the  information  (data  in  context)  necessary  to 
support  command  decision  making  (see,  for  example,  MCCTS  2016).  So-called 
“clumsy  automation”  can  make  Soldier  tasks  more  and  not  less  demanding.  Clumsy 
automation  refers  to  automation  that  is  improperly  designed  or  does  not  take  into 
consideration  how  the  tasks  perfonned  in  a  cognitive  work  system  such  as  a  CP  are 
actually  performed  or  might  change  with  the  addition  of  that  automated  support 
(Woods  1996). 
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Appendix  A.  Army  Knowledge  Management  Introduction  to  the 

KM  Maturity  Model  Tool 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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A.l  Background/Purpose 

The  purpose  of  this  document  is  to  introduce  to  Army  Knowledge  Management 
(KM)  practitioners  a  tool  that  is  being  developed  to  facilitate  assessing  the  maturity 
of  a  KM  program.  KM  has  been  a  part  of  the  United  States  Anny  since  2004  with 
the  founding  of  the  Battle  Command  Knowledge  System  (BCKS)  at  the  Combined 
Arms  Center  at  Ft.  Leavenworth,  KS.  BCKS  as  an  organization  was  reorganized 
into  the  Army  Knowledge  Management  Proponent  Office  and  Anny  KM  doctrine 
has  matured  and  is  deeply  embedded  in  the  Mission  Command  Warfighting 
Function  as  a  key  staff  task  which  is  articulated  in  ADP  6.0  Mission  Command. 
Since  2009  the  Army  Knowledge  Management  Proponent  Office  has  trained 
approximately  500  students  in  the  KM  process  and  in  March  2015,  published  the 
Anny  Techniques  Publication  (ATP)  6-01.1,  Techniques  for  Effective  Knowledge 
Management.  As  the  Army  continues  to  expand  its  KM  footprint,  the  need  for  KM 
practitioner  tools  increases.  This  document  contains  an  overview  of  the  Maturity 
Model  tool  and  how  to  use  it. 


At  what  level  are  our  staff  processes? 


Level  2 
“Repeatable' 


Some  processes  are  repeatable,  possibly  with  consistent  results. 
Process  discipline  is  unlikely  to  be  rigorous. 


Un-documented  and  in  a  state  of  dynamic  change,  tending  to  be  driven  in 
an  ad  hoc,  uncontrolled  and  reactive  manner  by  users  or  events. 


we  get  to  the  next 


level? 


First,  the  idea  of  KM  Maturity  is  not  new.  There  are  several  models  developed  in 
the  private  sector,  namely  by  two  organizations:  Camegie-Melon  and  American 
Productivity  and  Quality  Center  (APQC).  Both  of  these  models  have  made  their 
way  into  various  Army  organizations  and  are  undergoing  their  own  “pilot”  as  well. 
In  2009,  the  US  Army  War  College  published  a  paper  addressing  the  need  for  an 
Anny  version  of  a  KM  Maturity  Model  and  with  it  a  concept  of  what  it  might  look 
like.  Using  that  document  as  a  basis,  the  AKM  Maturity  model  was  expanded  as 
described  in  this  paper  and  when  used  by  a  unit  will  provide  a  baseline  assessment 
of  a  KM  program.  Second,  the  objective  of  this  tool  is  to  provide  an  organizational 
aid  to  improving  organizational  effectiveness  and  is  not  intended  as  an  inspection 
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item.  Most  importantly,  the  tool  is  designed  as  an  enabler  for  the  KM  team  to  help 
the  organization  execute  Mission  Command  more  effectively. 

A.2  How  to  Use  the  Maturity  Model  Tool 

The  tool  has  a  list  of  “efforts”  that  can  be  scored  using  a  simple  Likert  Scale  of  1— 
5  (an  effort  is  a  term  borrowed  from  the  Mission  Command  Information  System 
called  Command  Post  of  the  Future).  Each  effort  is  grouped  under  the  four 
components  of  KM:  people,  process,  tools,  and  organization  and  there  are  a  total  of 
42  efforts.  Efforts  may  be  related  to  other  efforts  in  other  components  as  well.  TAB 
1  of  the  tool  contains  a  description  of  what  the  effort  is  and  TAB  2  contains 
information  on  how  each  effort  contributes  to  KM  maturity,  as  well  as  suggested 
reference  material  that  will  aid  in  improving  the  score. 

For  example,  an  effort  listed  under  the  people  component  is  called  “Terms  of 
Reference”  (TOR).  A  TOR  document  is  an  emerging  TTP  that  contains  a  short 
summary  of  key  people  within  an  organization  and  their  specific  roles  and 
responsibilities  within  the  command  structure.  This  contributes  to  KM  maturity  in 
that  a  well-developed  TOR  document  eliminates  any  confusion  as  to  who  reports 
to  whom  and  more  specifically,  enhances  shared  understanding  regarding  roles  and 
responsibilities.  It  is  an  effective  emerging  best  practice. 


A.3  Maturity  Tool  Worksheet 


KM  Maturity  Model  Worksheet  (vl.O) 

KM  Principles:  Undertand,  Share,  Integrate,  Connect,  Le 


Rani 


C  Score 

D 

Tools 

Oreanization/Culture 

IM  Tools  '  2 

Culture  of  Col  laboration 

Collaboration  Tools  2 

Communities  of  Purpose 

Lessons  Learned  Repositories  2 

Organizational  Trust 

KM  SOP  *  2 

KM  Vision 

KM  Strategy  ^  2 

Learning  Environment 

Validated  COP  ^  2 

Cohesive  Teams 

KM  Planning  (Annex  Q)  2 

Social  Interaction 

Battle  Rhythm  ^  2 

Organizational  Feedback 

Working  Groups  and  Boards  2 

Mission  Command 

PACE  Plan  *  2 

KM  Chain  of  Command 

Learning  Tools  2 

Expertise  Location  System  2 

Virtual  Communities  2 

ORG  SOPS/Policies  *  2 

Column  Total  (Max  =70  Points)  28 

Column  Total  (Max  =50  Points) 

Score 


People 

Terms  of  Reference 

KM  Roles  and  Responsibilities 

Battle  Rosters 

KMWG 

External  SMEs 
KMRs 


Existence  of  a  Terms  of 
Reference  Document 
and  frequency  of 
update  (i.e.  each 
mission) 

T 


ews 

als 


KM/Statt  Hehea' 

2  KM  Process 
2  KM/IM  Tools  Training 
CCIR/SIR  Management 
Operations  Process 
COP/Dashboard  Management 
Time  Management 
Lessons  Learned  Management 
Task  Management 
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A.4  The  Rating  Scale 

The  Likert  scale  of  1-5  is  used  as  a  way  to  standardize  and  quantify  the  ratings.  The 
scale  is  universal  to  every  effort,  meaning  a  separate  set  of  metrics  is  not  needed 
for  each  effort.  Either  you  are  doing  it,  doing  it  to  some  level  of  proficiency 
(substitute  the  term  maturity  here),  or  you  are  not  doing  it  at  all.  Essentially,  all  the 
efforts  identified  were  chosen  because  they  were  either  a  doctrinal  requirement  or 
an  emerging  “KM  best  practice”.  It  should  be  noted  that  Anny  units  at  all  levels, 
including  generating  and  operating  forces,  are  required  to  execute  Mission 
Command.  Each  effort  is  linked  to  Mission  Command  doctrinally.  Based  on  that, 
the  tool  helps  to  identify  and  assess  activities  related  to  KM  that  the  organization 
needs  to  be  doing  well,  particularly  as  it  relates  to  the  execution  of  Mission 
Command.  The  scale  is  intended  to  answer  three  key  questions: 

1)  Is  the  unit  even  doing  this  effort? 

2)  If  so,  is  it  in  some  stage  of  development? 

3)  Is  it  codified  and  validated  in  daily  practices? 

All  efforts  are  scored  using  the  below  rating  scale: 


Rating 

d-5) 

1 

2 

3 

4 

5 


Description/Metric 


Unit  is  not  doing  this  effort  at  all 
Unit  has  started  this  effort  but  is  still  in  development 
Effort  is  well  developed  but  not  fully  implemented 
Unit  does  this  effort  and  it  is  fully  implemented 
Unit  has  assessed  the  effort  and  has  validated  its  utility 


Minimum 


Maximum 


To  use  the  tool,  open  the  spreadsheet  and  click  on  Tab  1-  KM  Maturity  Worksheet. 
Rate  each  effort  using  the  dropdown  menu  (instructions  are  on  the  lower  left). 
Evaluate  all  42  efforts  and  score  them  individually.  The  goal  of  the  tool  is  to  end 
up  with  an  overall  maturity  rating  as  a  baseline.  This  can  be  done  by  an  individual 
Knowledge  Manager,  or  collectively  with  the  Knowledge  Management  Working 
Group  (KMWG).  The  tool  automatically  calculates  the  total  maturity  for  each  KM 
component,  then  provides  an  overall  maturity  score  of  1-5. 

To  find  additional  information  about  each  effort  and  its  relationship  to  KM,  click 
on  Tab-2  References.  Scroll  down  to  find  the  effort  you  need  additional 
information.  On  the  far  right  column  is  a  list  of  potential  resources  that  you  can 
research  about  the  effort. 
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A.5  What  You  Should  Do  with  the  Results 


Every  unit  has  to  start  their  KM  program  somewhere,  and  some  of  the  efforts  are 
geared  towards  getting  the  program  off  the  ground  (i.e.  establishing  a  KMWG,  or 
developing  a  KM  SOP).  For  many  of  the  efforts,  the  unit  is  already  doing  the  effort 
(i.e.  managing  CCIRs,  or  developing  a  Common  Operating  Picture).  This  tool 
allows  the  KM  team  to  assess  and  quantify  how  well.  The  five  step  KM  process 
starts  with  assess,  (followed  by  design,  develop,  pilot,  and  implement).  Intended 
use  of  this  tool  is  in  the  assessment  phase.  Therefore,  if  the  KM  program  is  just 
getting  started,  the  focus  should  be  on  the  efforts  that  are  specific  to  implementing 
the  program.  Use  the  results  from  this  initial  assessment  to  develop  a  priority  list  to 
improve  the  unit’s  ability  to  more  effectively  execute  Mission  Command.  Then 
periodically  conduct  reassessments.  Finally,  enter  the  date  and  the  score  on  the  right 
hand  side  of  the  sheet  to  mark  progress.  Good  luck! 


Unit/Organization 


KM  COMPONENT  SCORE 

MAX  SCORE 

Ml 

People 

15 

30 

2.5 

Process 

25 

60 

2.08333 

Tools 

35 

70 

2.5 

Organization/Culture 

21 

50 

2.1 

TOTAL  KM  MATURITY  SCORE 

96 

190 

51% 
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Appendix  B.  Post  Knowledge  Management  Representative 
Course  Training  Evaluation  Questionnaire  Knowledge 
Management  Training  Experience 


This  appendix  appears  in  its  original  form,  without  editorial  change. 
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Quality  of  experience 

Strongly 

disagree 

Disagree 

Neither 
agree  nor 
disagree 

Agree 

Strongly 

agree 

1. 1  now  have  a  better 
understanding  of  what  knowledge 
management  (KM)  provides 

1 

2 

3 

4 

5 

2. 1  better  understand  how  the 
people,  processes,  and  tool 
components  of  KM  support  the 
organization. 

1 

2 

3 

4 

5 

3. 1  better  understand  how 
knowledge  flows  to  create  shared 
understanding. 

1 

2 

3 

4 

5 

4. 1  better  understand  how  KM 
supports  the  operations  process 

1 

2 

3 

4 

5 

5. 1  better  understand  how  to 
apply  the  KM  process  within  my 
staff  position  and  role. 

1 

2 

3 

4 

5 

6. 1  better  understand  the  different 
roles  and  responsibilities  of 
people  (Soldier,  KMO,  KMR, 

CoS,  etc.)  in  the  KM  process. 

1 

2 

3 

4 

5 

7. 1  better  understand  vertical  and 
horizontal  collaboration. 

1 

2 

3 

4 

5 

8. 1  believe  this  training  will 
enable  better  information  sharing 
throughout  my  unit. 

1 

2 

3 

4 

5 

9. 1  better  understand  how  to 
leverage  tools  that  enable 
information  sharing. 

1 

2 

3 

4 

5 

10. 1  believe  I  can  use  the 
knowledge  gained  in  training  to 
assist  in  creating,  executing,  and 
evaluating  a  KM  unit  plan. 

1 

2 

3 

4 

5 

1 1 . 1  believe  training  prepared  me 
to  use  the  KM  maturity  model  to 
conduct  a  unit  KM  assessment. 

1 

2 

3 

4 

5 

12. 1  believe  the  KM  maturity 
model  can  assist  me  in  identifying 
knowledge  sharing  weaknesses  in 
my  unit. 

1 

2 

3 

4 

5 

13.1  believe  my  leaders  will 
embrace  KM  to  better  our  unit 
shared  understanding  and 
collaboration. 

1 

2 

3 

4 

5 
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List  of  Symbols,  Abbreviations,  and  Acronyms 


2/1  AD  2nd  Brigade  Combat  Team  of  the  1st  Armored  Division 

ABCA  American,  British,  Canadian,  Australian,  and  New  Zealand  Armies 

Program 

ARL  US  Army  Research  Laboratory 

AWA  Anny  Warfighting  Assessment 

BCKS  Battle  Command  Knowledge  System 

BDE  brigade 

BMC  Brigade  Modernization  Command 

CDR  commander 

CO  commanding  officer 

COIN  counterinsurgency 

COP  Common  Operating  Picture 

CP  command  post 

CTC  Combat  Training  Center 

C W 2  Chief  W arrant  Officer  2 

DA  Department  of  the  Army 

DOTLP  Doctrine,  Organization,  Training,  Leadership  and  Education,  and 

Personnel 

FA  Functional  Area 

FAA  Federal  Aviation  Administration 

HSI  Human-Systems  Integration 

HRED  Human  Research  and  Engineering  Directorate 

IM  information  management 

IT  information  technology 

KM  knowledge  management 

KM/IM  knowledge  and  information  management 


Approved  for  public  release;  distribution  is  unlimited. 


37 


KMO 

Knowledge  Management  Officer 

KMR 

Knowledge  Management  Representative 

KMQC 

Knowledge  Management  Qualification  Course 

KMRC 

Knowledge  Management  Representative  Course 

KMWG 

Knowledge  Management  Working  Group 

MCCoE 

Mission  Command  Center  of  Excellence 

MIP 

Multilateral  Interoperability  Program 

MTOE 

Modified  Table  of  Organization  and  Equipment 

NATO 

North  Atlantic  Treaty  Organization 

NCO 

noncommissioned  officer 

NIE 

Network  Integration  Evaluation 

0-3 

captain 

0-4 

major 

0/C 

Observer/Controller 

S-3 

operations  and  training 

S-6 

signal 

SA 

Situation  Awareness 

SLEO 

Senior  Leader  Executive  Overview 

SOP 

standard  operating  procedure 

SME 

subject  matter  expert 

TOC 

Tactical  Operations  Center 

TOR 

Terms  of  Reference. 

TTPs 

tactics,  techniques,  and  procedures 

XO 

executive  officer 
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1 

DEFENSE  TECHNICAL 

1 

OSD  OUSD  ATL 

(PDF) 

INFORMATION  CTR 

(PDF) 

HPT&B  B  PETRO 

DTIC  OCA 

4800  MARK  CENTER  DRIVE 
SUITE  17E08 

2 

DIRARL 

ALEXANDRIA  VA  22350 

(PDF) 

RDRL  DCM 

IMAL  HRA  RECORDS  MGMT 

ABERDEEN  PROVING  GROUND 

RDRL  DCL 

TECH  LIB 

12 

DIRARL 

(PDF) 

RDRL  HR 

1 

GOVT  PRINTG  OFC 

J  LOCKETT 

(PDF) 

A  MALHOTRA 

P  FRANASZCZUK 

K  MCDOWELL 

1 

ARL 

KOIE 

(PDF) 

RDRL  HRB  B 

RDRL  HRB 

T  DAVIS 

D  HEADLEY 

BLDG  5400  RM  C242 

RDRL  HRB  BB 

REDSTONE  ARSENAL  AL 

J  K  HAWLEY 

35898-7290 

RDRL  HRB  C 

J  GRYNOVICKI 

8 

ARL 

RDRL  HRB  D 

(PDF) 

SFC  PAUL  RAY  SMITH 

C  PAULILLO 

CENTER 

RDRL  HRF  A 

RDRL  HRO  COL  H  BUHL 

A  DECOSTANZA 

RDRL  HRF  J  CHEN 

RDRL  HRF  B 

RDRL  HRA  I  MARTINEZ 

A  EVANS 

RDRL  HRR  R  SOTTILARE 

RDRL  HRF  C 

RDRL  HRA  C  A  RODRIGUEZ 

J  GASTON 

RDRL  HRA  B  G  GOODWIN 

RDRL  HRF  D 

RDRL  HRA  A  C  METEVIER 

RDRL  HRA  D  B  PETTIT 

12423  RESEARCH  PARKWAY 
ORLANDO  FL  32826 

A  MARATHE 

1 

USA  ARMY  G1 

(PDF) 

DAPE  HSI  B  KNAPP 

300  ARMY  PENTAGON 

RM  2C489 

WASHINGTON  DC  203 10-0300 

1 

USAF  711  HPW 

(PDF) 

7 1 1  HP  W/RH  KGEISS 

2698  GST  BLDG  190 

WRIGHT  PATTERSON  AFB  OH 
45433-7604 

1 

USN  ONR 

(PDF) 

ONR  CODE  341  J  TANGNEY 

875  N  RANDOLPH  STREET 

BLDG  87 

ARLINGTON  VA  22203-1986 

1 

USA  NSRDEC 

(PDF) 

RDNS  D  D  TAMILIO 

10  GENERAL  GREENE  AVE 

NATICK  MA  01760-2642 
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